A Scalable, Robust, and Decentralised Resource Discovery Service for Large Scale Federated Grids

Efficient Resource discovery mechanism is one of the fundamental requirement for Grid computing systems, as it aids in resource management and scheduling of applications. Resource discovery involves searching for resources that match the user’s application requirements. Various kinds of solutions to Grid resource discovery have been developed, including the centralised and hierarchical information server approach. However, these approaches have serious limitations in regards to scalability, fault-tolerance and network congestion. To overcome such limitations, we propose a decentralised Grid resource discovery system based on a spatial publish/subscribe index. It utilises a Distributed Hash Table (DHT) routing substrate for delegation of d-dimensional service messages. Our approach has been validated using a simulated publish/subscribe index that assigns regions of a d-dimensional resource attribute space to the grid peers in the system. We generated the resource attribute distribution using the configurations obtained from the Top 500 Supercomputer list. The simulation study takes into account various parameters such as resource query rate, index load distribution, number of index messages generated, overlay routing hops and system size. Our results show that grid resource query rate directly affects the performance of the decentralised resource discovery system, and that at higher rates the queries can experience considerable latencies. Further, contrary to what one can expect, system size does not have a significant impact on the performance of the system, in particular the query latency.